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Executive Summary 

This memo examines the outcomes and descriptions of the existing studies on K-12 online learning 
effectiveness and provides a literature review. 

There are a number of rigorous studies that have examined the question, "Is online learning 
effective?" However, there is not a single, large-scale, national study comparing students taking 
online courses with traditional students, using control groups in the instructional design. The most 
in-depth, large-scale study to date is a meta-analysis and review of online learning studies from the 
U.S. Department of Education. 

This memo contains three sections: 1) a summary of the major study by the U.S. Department of 
Education, 2) a brief literature review of online learning research and studies, and 3) future research 
recommendations. The conclusion of the meta-analysis of these studies is that online learning offers 
promising, new models of education that are effective. 



1. U.S. Department of Education Evaluation of Evidence- 
Based Practices in Online Learning: a Meta-Analysis 
and Review of Online Learning Studies 

The U.S. Department of Education released a meta-analysis and review of literature of 51 online 
learning studies in 2009. The overall results of the "meta-analysis found that, on average, students 
in online learning conditions performed better than those receiving face-to-face instruction" (U.S. 
Department of Education, 2009, p. ix). The study looked at studies comparing both online and 
blended learning environments to the face-to-face learning environment. In the studies focused on 
blended environments and face-to-face instruction, "blended instruction has been more effective, 
providing a rationale for the effort required to design and implement blended approaches (U.S. 
Department of Education, 2009, p. xvii). 

The main findings of the U.S. Department of Education study (2009) from the literature review was 
that: 



■ Few rigorous research studies of the effectiveness of online learning for K-12 students have 
been published. 
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The meta-analysis of 51 study effects, 44 of which were drawn from research with older learners, 
found that: 

■ Students who took all or part of their class online performed better, on average, than those 
taking the same course through traditional face-to-face instruction. 

■ Instruction combining online and face-to-face elements had a larger advantage relative to 
purely face-to-face instruction than did purely online instruction. 

■ Studies in which learners in the online condition spent more time on task than students in 
the face-to-face condition found a greater benefit for online learning. 

■ Most of the variations in the way in which different studies implemented online learning did 
not affect student learning outcomes significantly. 

■ The effectiveness of online learning approaches appears quite broad across different content 
and learner types. 

■ Effect sizes were larger for studies in which the online and face-to-face conditions varied 
in terms of curriculum materials and aspects of instructional approach in addition to the 
medium of instruction. 

■ The narrative review of experimental and quasi-experimental studies contrasting different 
online learning practices found that the majority of available studies suggest the following: 

■ Blended and purely online learning conditions implemented within a single study generally 
result in similar student learning outcomes. 

■ Elements such as video or online quizzes do not appear to influence the amount that 
students learn in online classes. 

■ Online learning can be enhanced by giving learners control of their interactions with media 
and prompting learner reflection. 

■ Providing guidance for learning for groups of students appears less successful than does 
using such mechanisms with individual learners (p. xiv - xv). 

2. Literature Review: Effectiveness of Online Teaching 
and Learning 

Following is a brief overview of the research on effectiveness of K-12 online teaching and learning. 
The primary question addressed in most studies is how students enrolled in computer-mediated, 
asynchronous, online teaching and learning courses compares in student achievement outcomes on 
standardized tests to students taught in a traditional, synchronous, face-to-face setting. 

From 1989-2004, there were 15 studies published that met strict criteria for internal experimental 
validity comparing online courses with conventional courses. Cavanaugh, Gillan, Hess and Blomeyer 
(2005) published the first meta-analysis of online education outcomes focused entirely on K-12 
education, The Effects of Distance Education on K-12 Student Outcomes: a Meta-Analysis. The 
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meta-analysis found that virtual instruction produced results measuring student achievement that 
were "as good or better than" traditional face-to-face instruction. 

In 2003-2006, the U.S. Department of Education funded rigorous studies of educational technology 
and online learning (including the West Virginia Virtual School evaluation for the online Spanish 
course) with control groups in the experimental design, focused on measuring student achievement 
outcomes. In the West Virginia Ed Pace study of the Virtual School, students in the online Spanish 
I courses learned Spanish I as well as their peers, and the Virtual School Spanish students out- 
performed some peers in Spanish II. The evaluation found that Virtual School Spanish students 
learned valuable technology skills. Data suggest the effective use of some key elements of the online 
course in a blended model is associated with more successful implementations and powerful student 
outcomes. 

The Florida TaxWatch report published in 2007, "A Comprehensive Assessment of Florida Virtual 
School" examines the efficacy and efficiency of the Florida Virtual School, which offers online 
courses and Advanced Placement courses for middle and high school students statewide. 

Florida TaxWatch is a nonprofit, known as the "watch dog" of citizen's tax dollars. Florida TaxWatch 
conducts independent research on government expenditures, public policies and programs to 
increase productivity and accountability of Florida's government. The Florida TaxWatch's Center 
for Educational Performance and Accountability conducted the research to assess whether Florida 
Virtual School (FLVS) offers an efficient, taxpayer-accountable alternative and supplemental system 
of education. 

A description of the study reads: "The study examined student demographics, achievement and 
cost-effectiveness, finding that during the 2004-05 and 2005-06 school years FLVS students 
consistently outperformed their counterparts in Florida's traditional middle and high schools on such 
measures as grades, Advanced Placement scores and FCAT scores. All FLVS teachers are certified, 
and their pay is tied to student performance, making FLVS the only true performance-based 
education system in the state. The study also found that FLVS is a bargain for Florida taxpayers. 

Largely because it has no expenses related to transportation or construction and maintenance of 
physical facilities, FLVS is able to offer computer-delivered instruction at a lower per-student cost 
than traditional schools." 

The four main findings of the study are: 

■ FLVS is a better use of taxpayer dollars compared to traditional education with results; 

■ FLVS students perform better than students in traditional classes, based on student 
achievement; 

■ FLVS is serving a higher proportion of minority and underserved students demographically 
statewide; 

■ FLVS provides a new, more rigorous model of accountability for K-12 public education that is 
data-rich and performance-driven. 
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A study by Lowes at Columbia University found that teachers' instructional practices are transformed 
by learning how to teach online in developing new skills and pedagogical strategies using 
technology (Lowes 2005). The research reported that online teaching improves practices in both 
virtual and face-to-face settings, and 75% of teachers said that teaching online had a positive 
impact on their face-to-face teaching. Lowes examined how online teachers can serve as reform 
agents in the schools where they also teach face-to-face courses. 

Course and instructional design are important considerations for online learning effectiveness. Well- 
structured courses have been shown to be a critical student success factor (Weiner 2003). Recent 
state and district virtual school reports have cited a range of student support services that contribute 
to increasing course completion rates (Harlow & Baenen 2003). 

Interaction is the heart of online learning. Teachers have reported that their interactions with 
students, parents and colleagues were more often focused on teaching and learning in online 
courses than in the traditional setting (Muirhead 2000). Interaction is named as the primary 
difference between online and face-to-face instruction and one of the most important aspects of 
the online setting (Weiner 2003). In virtual schools, participants seek both deeper and stronger 
relationships, and they also value frequent and timely responses to questions (Weiner 2003). 

In 2008, the U.S. Department of Education published a report on the evaluation of online programs. 
An evaluation conducted of the Washington State Digital Learning Commons (DLC) online courses 
was highlighted in the study. The research was clear: student access to DLC online courses increased 
on-time graduation rates and college/workforce readiness at the schools studied in Washington 
State. "Of the 115 students who graduated, 33% would not have graduated without a course made 
available through the Digital Learning Commons and 61% of the students who participated in the 
study took advanced classes to better prepare themselves for college. The Digital Learning Commons 
evaluation concluded that online courses were a new approach and recommended that future 
research be focused on the impact of online courses, building the evidence-base of outcomes and 
results that can be objectively gathered and tabulated to add to the research in this field. 

The first significant research study on the socialization of students in full-time, online public schools 
was published in 2009 by the Center for Research in Educational Policy (CREP) at the University 
of Memphis in collaboration with Interactive Education Systems Design (IESD). The results of this 
study provide substantial evidence supporting the conclusion that students enrolled in full-time, 
online public schools are at least as well socialized as equivalent students enrolled in traditional 
public schools. The study found that online students might have an advantage in their social skills 
development when they are highly engaged in activities outside the school day involving both peer 
interaction and activities not involving peer interaction (IESD, 2009). 

The small body of research focused on the effectiveness of K-12 virtual schooling programs supports 
findings of similar studies on online courses offered in higher education. The college-level studies 
find "no significant difference" in student performance in online courses versus traditional face- 
to-face courses, and in particular programs that students learning online are performing "equally 
well or better". Last year, the National Survey of Student Engagement (NSSE 2008) study found 
that online learners reported deeper approaches to learning than classroom-based learners and 
experienced "better use of higher order thinking skills, integrative thinking, and reflective learning." 
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Success. Innovations in Education. U.S. Department of Education. 



3. Future Research 

Larger-scale studies are needed to show the correlations between program models, instructional 
models, technologies, conditions and practices for effective online learning. 

To conduct state-wide research on different online programs, one recommended strategy for a 
baseline study would be to employ a researcher to collect existing data sets from standardized 
achievement tests from a state data system — student performance on tests, the correlating student 
demographics — and compare those with the virtual school student performance data in the 
state. This type of data is available at state departments of education in the 18 states with virtual 
charter schools that are required to comply with the state mandated No Child Left Behind tests for 
grades 3-8. Comparable achievement data would also be available in states with end-of-course 
testing, such as Georgia and Florida. A large-scale study examining comparisons between online 
and traditional students across states is difficult when there are different academic standards and 
assessments in each state. 



Conclusion 

Online learning has the potential to transform teaching and learning by redesigning traditional 
classroom instructional approaches, personalizing instruction and enhancing the quality of learning 
experiences. The preliminary research shows promise for online learning as an effective alternative 
for improving student performance across diverse groups of students. 
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